System and method for personalized preference optimization

ABSTRACT

A system and method is provided for using biometric data from an individual to determine at least one emotion, mood, physical state, or mental state (“state”) of the individual, which is then used, either alone or together with other data, to provide the individual with certain web-based data. In one embodiment of the present invention, a Web host is in communication with at least one network device, where each network device is operated by an individual and is configured to communicate biometric data of the individual to the Web host. The Web host is then configured to use the biometric data to determine at least one state of the individual. The determined state, either alone or together with other data (e.g., interest data), is then used to provide the individual with certain content (e.g., web-based data) or to perform a particular action.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority pursuant to 35 U.S.C. § 119(e) to U.S.Provisional Application No. 62/214,496, filed Sep. 4, 2015, whichapplication is specifically incorporated herein, in its entirety, byreference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to use of biometric data from anindividual to determine at least one emotional state, mood, physicalstate, or mental state (“state”) of the individual, either (i) at aparticular time or (ii) in response to at least one thing in a proximityof the individual at a time that the biometric data is being collected,and using the at least one state, either by itself or together withother data (e.g., data related to the at least one thing, interest data,at least one request (e.g., question, command, etc.), etc.) to provide aparticular response (e.g., provide certain web-based data to theindividual, perform a particular action, etc.).

2. Description of Related Art

Recently, devices have been developed that are capable of measuring,sensing, or estimating at least one metric related to a humancharacteristic, commonly referred to as biometric data. For example,devices that resemble watches have been developed that are capable ofmeasuring an individual's heart rate or pulse, and using that datatogether with other information (e.g., the individual's age, weight,etc.) to calculate a resultant, such as the total calories burned by theindividual in a given day. Similar devices have been developed formeasuring, sensing, or estimating other metrics, such as blood pressure,breathing patterns, and blood-alcohol level, or for the identificationor recognition of individuals in, amongst others, security applications,through the recording and analysis with devices such as iris scanners,or microphones with voice pattern recognition, of the individual'sunique biometric or physiological characteristics, to name a few. Thesedevices are generically referred to as biometric devices.

While the types of biometric devices continue to grow, the way in whichbiometric data is used remains relatively stagnant. For example, heartrate data is typically used to give an individual information on theirpulse and calories burned. By way of another example, eye movement datacan be used to determine whether and to what extent the individual isunder the influence of alcohol. By way of yet another example, anindividual's breathing pattern may be monitored by a doctor, nurse, ormedical technician to determine whether the individual suffers fromsleep apnea.

While biometric data is useful in and of itself, such data may alsoindicate how the individual is feeling (e.g., at least one emotionalstate, mood, physical state, or mental state) at a particular time or inresponse to the individual being in the presence of at least one thing(e.g., a person, a place, textual content (or words included therein ora subject matter thereof), video content (or a subject matter thereof),audio content (or words included therein or a subject matter thereof),etc.). Thus, it would be advantageous, and a need exists, for a systemand method that uses the determined state (e.g., emotion state, mood,physical state, or mental state), either alone or together with otherinformation (e.g., at least one thing, interest data, at least onerequest (e.g., question, command, etc.), etc.), to produce a certainresult, such as provide the individual with certain web-based content(e.g., a certain web page, a certain advertisement, etc.) and/or performat least one action. While providing a particular message to every knownbiometric state may not be reasonable for content creators to understandand target, human emotions and moods provide a specific context fortargeting messages that is easily understood by content creators.

SUMMARY OF THE INVENTION

The present invention is directed toward personalization preferenceoptimization, or to the use of biometric data from an individual todetermine at least one emotional state, mood, physical state, or mentalstate (“state”) of the individual, which is then used, either alone ortogether with other data (e.g., at least one thing in a proximity of theindividual at a time that the individual is experiencing the emotion,interest data from a source of web-based data (e.g., bid data, etc.),etc.) to provide the individual with certain web-based data or toperform a particular action.

Preferred embodiments of the present invention operate in accordancewith a Web host in communication with a plurality of content providers(i.e., sources) and at least one network device via a wide area network(WAN), wherein the network device is operated by an individual and isconfigured to communicate biometric data of the individual to the Webhost. The content providers provide the Web host with content, such aswebsites, web pages, image data, video data, audio data, advertisements,etc. The Web host is then configured to receive biometric data from thenetwork device, where the biometric data is acquired from and/orassociated with an individual that is operating the network device. Anapplication is then used to determine at least one emotion, mood,physical state, or mental state from the received biometric data. Thisis done using known algorithms and/or correlations between biometricdata and various states, as stored in the memory device.

In one embodiment of the present invention, content providers mayexpress interest in providing the web-based data to an individual in aparticular emotional state. In another embodiment of the presentinvention, content providers may express interest in providing theweb-based data to an individual or other concerned party (such asfriends, employer, care provider, etc.) that experienced a particularemotion in response to a thing (e.g., a person, a place, a subjectmatter of textual content, a subject matter of video content, a subjectmatter of audio content, etc.). The interest may be a simple “Yes” or“No,” or may be more complex, like interest on a scale of 1-10, anamount the content owner is willing to pay per impression (CPM), or anamount the content owner is willing to pay per click (CPC).

The interest data, alone or in conjunction with other data (e.g.,randomness, demographics, etc.), may be used by the application todetermine content data (e.g., an advertisement, etc.) that should beprovided to the individual. For example, if the interest data includesdifferent bids for a particular emotion or an emotion-thingrelationship, the application may provide the advertisement with thehighest bid to the individual that experienced the emotion. In otherembodiments, other data is taken into consideration in providing contentto the individual. In these embodiments, at least interest data is takeninto account in selecting the content that is to be provided to theindividual.

In one method of the present invention, biometric data is received froman individual and used to determine a corresponding emotion of theindividual, such as happiness, anger, surprise, sadness, disgust, orfear. It is to be understood that emotional categorization ishierarchical and that such a method may allow targeting more specificemotions such as ecstasy, amusement, or relief, which are all subsets ofthe emotion of joy. A determination is made as to whether the emotion isthe individual's current state, or whether it is based on theindividual's response to a thing (e.g., a person, place, informationdisplayed to the individual, etc.). If the emotion is the individual'scurrent state, then content is selected based on at least theindividual's current emotional state and interest data. If, however, theemotion is the individual's response to a thing, then content isselected based on at least the individual's emotional response to thething (or subject matter thereof) and interest data. The selectedcontent is then provided to the individual, or network device operatedby the individual.

Emotion, mood, physical, or mental state of an individual can also betaken into consideration when performing a particular action or carryingout a particular request (e.g., question, command, etc.). In otherwords, prior to performing a particular action (e.g., under thedirection of an individual, etc.), a network-connected or network-awaresystem or device may take into consideration an emotion, mood, physical,or mental state of the individual. For example, a command or instructionprovided by the individual, either alone or together with otherbiometric data related to or from the individual, may be analyzed todeterminate the individual's current mood, emotional, physical, ormental state. The network-connected or network-aware system or devicemay then take the individual's state into consideration when carryingout the command or instruction. Depending on the individual's state, thesystem or device may warn the individual before performing the requestedaction, or may perform another action, either in additional to orinstead of the requested action. For example, if it is determined that adriver of a vehicle is angry or intoxicated, the vehicle may provide thedriver with a warning before starting the engine, may limit maximumspeed, or may prevent the driver from operating the vehicle (e.g.,switch to autonomous mode, etc.).

A more complete understanding of a system and method for using biometricdata to determine at least one emotional state, mood, physical state, ormental state (“state”) of an individual, wherein at least said state isused to provide or perform a particular result, will be afforded tothose skilled in the art, as well as a realization of additionaladvantages and objects thereof, by a consideration of the followingdetailed description of the preferred embodiment. Reference will be madeto the appended sheets of drawings, which will first be describedbriefly.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a Web host in communication with at least one contentprovider and at least one network device via a wide area network (WAN),wherein said Web host is configured to provide certain content to thenetwork device in response to biometric data (or data related thereto),as received from the network device;

FIG. 2 illustrates one embodiment of the Web host depicted in FIG. 1;

FIG. 3 provides an exemplary chart that links different biometric datato different emotions;

FIG. 4 provides an exemplary chart that links different responses todifferent emotions, different things, and different interest levels inthe same;

FIG. 5 illustrates a method in accordance with one embodiment of thepresent invention of using biometric data from an individual todetermine at least one emotion of the individual, and using the at leastone emotion, either alone or in conjunction with other data, to selectcontent to be provided to the individual;

FIG. 6 provides an exemplary biometric-sensor data string in accordancewith one embodiment of the present invention;

FIG. 7 provides an exemplary emotional-response data string inaccordance with one embodiment of the present invention;

FIG. 8 provides an exemplary emotion-thing data string in accordancewith one embodiment of the present invention;

FIG. 9 provides an exemplary thing data string in accordance with oneembodiment of the present invention;

FIG. 10 illustrates a network-enabled device that is in communicationwith a plurality of remote devices via a wide area network (WAN) and isconfigured to use biometric data to determine at least one state of anindividual and use the at least one state to perform at least oneaction;

FIG. 11 illustrates one embodiment of the network-enabled devicedepicted in FIG. 10; and

FIG. 12 illustrates a method in accordance with one embodiment of thepresent invention of using biometric data from an individual todetermine at least one state of the individual, and using the at leastone state to perform at least one action.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

The present invention is described as personalization preferenceoptimization, or using at least one emotional state, mood, physicalstate, or mental state (“state”) of an individual (e.g., determinedusing biometric data from the individual, etc.) to determine a response,which may include web-based data that is provided to the individual as aresult of the at least one state, either alone or together with otherdata (e.g., at least one thing (or data related thereto) in a proximityof the individual at a time that the individual is experiencing the atleast one emotion, etc.).

As shown in FIG. 1, preferred embodiments of the present inventionoperate in accordance with a Web host 102 in communication with at leastcontent provider (e.g., provider of web-based data) 104 and at least onenetwork device 106 via a wide area network (WAN) 100, wherein eachnetwork device 106 is operated by an individual and is configured tocommunicate biometric data of the individual to the Web host 102, wherethe biometric data is acquired using at least one biometric sensor 108.

While FIG. 1 depicts the preferred embodiment, it should be appreciatedthat other embodiments are within the spirit and scope of the presentinvention. For example, the network device 106 itself may be configuredto collect (e.g., sense, etc.) biometric data on the individual. Thismay be accomplished, for example, through the use of at least onemicrophone (e.g., to acquire voice data from the individual), at leastone camera (e.g., to acquire video data on the individual), at least oneheart rate sensor (e.g., to measure heart rate data on the individual),at least one breath sensor (e.g., to measure breath chemical compositionof the individual), etc. By way of another example, the host may beconfigured to communicate directly with the network device, for exampleusing a wireless protocol such as Bluetooth, Wi-Fi, etc. By way of yetanother example, the host may be configured to acquire biometric datadirectly from the individual using, for example, at least onemicrophone, at least one camera, or at least one sensor (e.g., a heartrate sensor, a breath sensor, etc.). In this example, the host may beconfigured to provide data to the individual (e.g., display data on ahost display) or perform at least one action (e.g., switch an automobileto autopilot, restrict speed, etc.).

With reference to FIGS. 1 and 2, the content provider 104 provides theWeb host 102 with web-based data, such as a website, a web page, imagedata, video data, audio data, an advertisement, etc. Other web-baseddata is further provided to the Web host 102 by at least one othercontent provider (not shown). The plurality of web-based data (e.g.,plurality of websites, plurality of web pages, plurality of image data,plurality of video data, plurality of audio data, plurality ofadvertisements, etc.) is stored in a memory device 204 along with otherdata (discussed below), such as information that links differentbiometric data to different states (see FIG. 3) and interest data (seeFIG. 4). It should be appreciated that the present invention is notlimited to the memory device 204 depicted in FIG. 2, and may includeadditional memory devices (e.g., databases, etc.), internal and/orexternal to the Web host 102.

The Web host 102 is then configured to receive biometric data from thenetwork device 106. As discussed above, the biometric data is preferablyrelated to (i.e., acquired from) an individual who is operating thenetwork device 106, and may be received using at least one biometricsensor 108, such as an external heart rate sensor, etc. As discussedabove, the present invention is not limited to the biometric sensor 108depicted in FIG. 1, and may include additional (or different) biometricsensors (or the like, such as microphones, cameras, etc.) that areexternal to the network device 106, and/or at least one biometric sensor(or the like, such as microphones, cameras, etc.) internal to thenetwork device. If the biometric sensor is external to the networkdevice, it may communicate with the network device via at least one wireand/or wirelessly (e.g., Bluetooth, Wi-Fi, etc.).

It should be appreciated that the present invention is not limited toany particular type of biometric data, and may include, for example,heart rate, blood pressure, breathing rate, temperature, eye dilation,eye movement, facial expressions, speech pitch, auditory changes, bodymovement, posture, blood hormonal levels, urine chemical concentrations,breath chemical composition, saliva chemical composition, and/or anyother types of measurable physical or biological characteristics of theindividual. The biometric data may be a particular value (e.g., aparticular heart rate, etc.) or a change in value (e.g., a change inheart rate), and may be related to more than one characteristic (e.g.,heart rate and breathing rate).

It should also be appreciated that while best results come from directmeasurement of known individuals, the same methods of correlation can beapplied to general categories of people. An example is that a facialrecognition system may know that 90% of the people at a particularlocation, such as a hospital, are fearful and that an individual isknown to be at that location. Even if biometric data of that individualis not shared with the system, the correlation may be applied,preserving privacy and still allowing for statistically significanttargeting. Another example would be a bar that had urine chemicalanalyzers integrated into the bathrooms, providing general informationabout people at the bar. This data could then be coordinated with timeand location back to a group of people and provide significantcorrelations for targeting messages to an individual (e.g., anindividual who was at the bar during that time).

As shown in FIG. 2, the Web host 102 includes an application 208 that isconfigured to determine at least one state from the received biometricdata. This is done using known algorithms and/or correlations betweenbiometric data and different states, such as emotional states, as storedin the memory device 204. For example, as shown in FIG. 3, if thebiometric data 302 indicates that the individual is smiling (e.g., viause of at least one camera), then it may be determined that theindividual is experiencing the emotion 304 of happiness. By way of otherexamples, if the biometric data 302 indicates that the individual'sheart rate is steadily increasing (e.g., via use of a heart ratesensor), then it may be determined that the individual is experiencingthe emotion 304 of anger. If the biometric data 302 indicates that theindividual's heart rate temporarily increases (e.g., via use of a heartrate sensor), then it may be determined that the individual isexperiencing the emotion 304 of surprise. If the biometric data 302indicates that the individual is frowning (e.g., via use of at least onecamera), then it may be determined that the individual is experiencingthe emotion 304 of sadness. If the biometric data 302 indicates that theindividual's nostrils are flaring (e.g., via use of at least onecamera), then it may be determined that the individual is experiencingthe emotion 304 of disgust. And if the biometric data 302 indicates thatthe individual's voice is shaky (e.g., via use of at least onemicrophone), then it may be determined that the individual isexperiencing the emotion 304 of fear.

Information that correlates different biometric data to differentemotions or the like can come from different sources. For example, theinformation could be based on laboratory results, self-reporting trials,and secondary knowledge of emotions (e.g., the individual's use ofemoticons and/or words in their communications). Because someinformation is more reliable than other information, certain informationmay be weighted more heavily than other information. For example, incertain embodiments, clinical data is weighted heavier thanself-reported data. In other embodiments, self-reported data is weightedheavier than clinical data. Laboratory (or learned) results may includedata from artificial neural networks, C4.5, classification and/orregression trees, decision trees, deep learning, dimensionalityreduction, elastic nets, ensemble learning, expectation maximization,k-means, k-nearest neighbor, kernel density estimation, kernel principlecomponents analysis, linear regression, logical regression, matrixfactorization, naïve bayes, neighbor techniques, partial least squaresregression, random forest, ridge regression, support vector machines,multiple regression and/or all other learning techniques generally knownto those skilled in the art.

Self-reported data may include data where an individual identifies theircurrent state, allowing biometric data to be customized for thatindividual. For example, computational linguistics could be used toidentify not only what an individual is saying but how they are sayingit. In other words, the present invention could be used to analyze andchart speech patterns associated with an individual (e.g., allowing theinvention to determine who is speaking) and speech patterns associatedwith how the individual is feeling. For example, in response to “how areyou feeling today,” the user may state “right now I am happy,” or “rightnow I am sad.” Computational linguistics could be used to chartdifferences in the individual's voice depending on the individual'scurrent emotional state, mood, physical state, or mental state. Becausethis data may vary from individual to individual, it is a form ofself-reported data, and referred to herein as personalized artificialintelligence. The accuracy of such data, learned about the individual'sstate through analysis of the individual's voice (and then throughcomparison both to the system's historical knowledge base of states ofthe individual acquired and stored over time and to a potential widerdatabase of other users' states as defined by analysis of their voice),can further be corroborated and or improved, through cross-referencingthe individual's self-reported data with other biometric data, such asheart rate data, etc., when a particular state is self-reported anddetected and recorded by the system onto its state profile database.

The collected data, which is essentially a speech/mood profile for theindividual (a form of ID which is essentially the individual's uniquestate profile), can be used by the system that gathered the biometricdata or shared with other systems (e.g., the individual's smartphone,the individual's automobile, a voice or otherwise biometrically-enableddevice or appliance (including Internet of Things (IOT) devices or IOTsystem control devices), Internet or “cloud” storage, or any other voiceor otherwise biometrically-enabled computing or robotic device orcomputer operating system with the capability of interaction with theindividual, including but not limited to devices which operate usingvoice interface systems such as Apple's Siri, Google Assistant,Microsoft Cortana, Amazon's Alexa, and their successor systems). Becausethe shared information is unique to an individual, and can be used toidentify a current state of the individual, it is referred to herein aspersonalized artificial intelligence ID, or “PAIID.” In one embodimentof the present invention, the self-reported data can be thought of ascalibration data, or data that can be used to check, adjust, orcorrelate certain speech patterns of an individual with at least onestate (e.g., at least one emotion, at least one mood, at least onephysical state, or at least one mental state). The knowledge andpredictive nature inherent in the PAIID will be continuously improvedthrough the application of deep learning methodology with data labellingand regression as well as other techniques apparent to those skilled inthe art.

With respect to computational linguistics, it should be appreciated thatthe present invention goes beyond using simple voice analysis toidentify a specific individual or what the individual is saying.Instead, the present invention can use computational linguistics toanalyze how the individual is audibly expressing himself/herself todetect and determine at least one state, and use this determination asan element in providing content to the user or in performing at leastone action (e.g., an action requested by the user, etc.).

It should be appreciated that the present invention is not limited tousing a single physical or biological feature (e.g., one set ofbiometric data) to determine the individual's state. Thus, for example,eye dilation, facial expressions, and heart rate could be used todetermine that the individual is surprised. It should also beappreciated that an individual may experience more than one state at atime, and that the received biometric data could be used to identifymore than one state, and a system could use their analysis of theindividual's state or combination of states to assist it in deciding howbest to respond, for example, to a user request, or a user instruction,or indeed whether to do so at all. It should further be appreciated thatthe present invention is not limited to the six emotions listed in FIG.3 (i.e., happiness, anger, surprise, sadness, disgust, and fear), andcould be used to identify other (or alternate) emotional states, such asregret, love, anxiousness, etc. Finally, the present invention is notlimited to the application 208 as shown in FIG. 2, and may include oneor more applications operating on the Web host 102 and/or the networkdevice 106. For example, an application or program operating on thenetwork device 106 could use the biometric data to determine theindividual's emotional state, with the emotional state beingcommunicated to the Web host 102 via the WAN 100.

Despite preferred embodiments, the present invention is not limited tothe use of biometric data (e.g., gathered using sensors, microphones,and/or cameras) solely to determine an individual's current emotionalstate or mood. For example, an individual's speech (either alone or incombination with other biometric data, such as the individual's bloodpressure, heart rate, etc.) could be used to determine the individual'scurrent physical and/or mental health. Examples of physical healthinclude how an individual feels, such as healthy, good, poor, tired,exhausted, sore, achy, and sick (including symptoms thereof, such asfever, headache, sore throat, congested, etc.), and examples of mentalhealth include mental states, such as clear-headed, tired, confused,dizzy, lethargic, disoriented, and intoxicated. By way of example,computational linguistics could be used to correlate speech patterns toat least one physical and/or mental state. This can be done using eitherself-reported data (e.g., analyzing an individual's speech when theindividual states that they are feeling fine, under the weather,confused, etc.), general data that links such biometric data to physicaland/or mental state (e.g., data that correlates speech patterns (ingeneral) to at least one physical and/or mental states), or acombination thereof. Such a system could be used, for example, in ahospital to determine a patient's current physical and/or mental state,and provide additional information outside the standard physiological orbiometric markers currently utilized in patient or hospital care. If thephysical and/or mental state is above/below normal (N), which mayinclude a certain tolerance (T) in either direction (e.g., N+/−T)through the patient making a request or statement, or through responseto a question generated by the system, a nurse or other medical staffmember may be notified. This would have benefits such as providing anadditional level of patient observation automation or providing earlywarning alerts or reassurance about the patient through system analysisof their state.

As shown in FIG. 2, the Web host 102 may also include other components,such as a keyboard 210, allowing a user to enter data, a display 206,allowing the Web host 102 to display information to the user (orindividual in embodiments where the biometric sensors are internal tothe Web Host 102), a transceiver 212, allowing the Web host 102 tocommunicate with external devices (e.g., the network device 106 via theWAN 100, the network device 106 via a wireless protocol, an externalbiometric sensor via a wireless protocol, etc.), and a processor 202,which may control the reception and/or transmission of information tointernal and/or external devices and/or run the application 208, ormachine-readable instructions related thereto.

In one embodiment of the present invention, a source of web-based data(e.g., content provider) may express interest in providing the web-baseddata to an individual in a particular emotional state. For example, asshown in FIG. 4, an owner of feel-good content (e.g., kittens inhumorous situations, etc.) may express an interest in providing thecontent to individuals who are currently feeling the emotion of sadness.The interest may be as simple as “Yes” or “No,” or may be more complex,like interest on a scale of 1-10. In another embodiment of the presentinvention, a source of web-based data may express interest in providingthe web-based data to an individual that experienced a particularemotion in response to a thing (e.g., a person, a place, a subjectmatter of textual data, a subject matter of video data, a subject matterof audio data, etc.). For example, as shown in FIG. 4, an owner of amatchmaking service may express an interest ($2.50 CPM) in providing arelated advertisement to individuals, their friends, or their contactsthat experienced the emotion of happiness when they are in closeproximity to a wedding (thing) (e.g., being at a wedding chapel, readingan email about a wedding, seeing a wedding video, etc.). By way ofanother example, an owner of a jewelry store may express an interest(5.00 CPC) in providing an advertisement to individuals that experiencedthe emotion of excitement when they are in close proximity to a diamond(thing) (e.g., being at a store that sells diamonds, reading an emailabout diamonds, etc. The selection of web-based content and/or interestmay also be based on other data (e.g., demographic data, profile data,click-through responses, etc.). Again, the interest may be a simple“Yes” or “No,” or may be more complex, like an interest on a scale of1-10, an amount an owner/source of the content is willing to pay perimpression (CPM), or an amount an owner/source of the content is willingto pay per click (CPC).

Another embodiment of the invention may involve a system integrated withat least one assistance system, such as voice controls orbiometric-security systems, where the emotionally selected messages areprimarily warnings or safety suggestions, and are only advertisements inspecific relevant situations (discussed in more detail below). Anexample would be of a user who is using a speech recognition system toreceive driving directions where the user's pulse and voice dataindicate anger. In this case, the invention may tailor results to benearby calming places and may even deliver a mild warning that accidentsare more common for agitated drivers. This is an example where theprimary purpose of the use is not the detection of emotion, but theemotion data can be gleaned from such systems and used to targetmessages to the individual, contacts, care-providers, employers, or evenother computer systems that subscribe to emotional content data. Analternate example would be a security system that uses retinal scanningto identify pulse and blood pressure. If the biometric data correlatesto sadness, the system could target the individual with uplifting orpositive messages to their connected communication device or even alerta care-provider. In other instances, for example with a vehicle equippedwith an autonomous driving system, based on the system's analysis of thebiometric feedback of the individual, the driving system could advise onexercising caution or taking other action in the interests of the driverand others (e.g., passengers, drivers of other vehicles, etc.).

It should be noted that in this invention some use cases theindividual's private data is provided with the users consent to thesystem, but in many cases the emotional response could be associatedwith a time-of-day, a place, or a given thing (e.g., jewelry shop,etc.), so personally identifying information (PII) does not need to beshared with the message provider. In the example of a jewelry shop, thesystem simply targets individuals and their friends with strong joycorrelations. While in certain embodiments, individuals may be offeredthe opportunity to share their PII with message providers, the systemcan function without this level of information.

The interest data, and perhaps other data (e.g., randomness,demographics, etc.) may be used by the application (FIG. 2 at 208) todetermine web-based data (e.g., an advertisement, etc.) that should beprovided to the individual. For example, if the interest data includesdifferent bids for a particular emotion or an emotion-thingrelationship, the application may provide the advertisement associatedwith the highest bid to the individual (or related network device) whoexperienced the emotion. In other embodiments, other data is taken intoconsideration in providing web-based data to the individual. In theseembodiments, interest data is but one criteria that is taken intoaccount in selecting web-based data that is provided to the individual.

It should be appreciated that the “response” to an individual in aparticular state, or having an emotional response to a thing, is notlimited to providing the individual with web-based content, and mayinclude any action consistent with the determined state. In other words,the determined state can be used by the host (e.g., automobile,smartphone, etc.) to determine context, referred to herein as“situational context.” For example, as shown in FIG. 10, an automobile1002 may include a host 1004 that determines (using biometric dataacquired via a camera, microphone, or sensor) that the driver (notshown) is impaired or emotional (e.g., angry, excited, etc.), may switchto auto-pilot, or may limit the maximum speed of the vehicle. In thisembodiment, the “response” carried out by the host may be based oncommands provided by the individual (e.g., verbal or otherwise) and atleast one emotion or mood of the individual, where the emotion/mood isdetermined based on biometric data. For example, where a voice commandto perform an action (by itself) may result in a robot performing anaction at a normal pace (which may have the benefit of batterypreservation, accuracy, etc.), a voice command to perform the sameaction along with biometric data expressing a mood of urgency may resultin the robot performing the action at a quicker pace.

In one embodiment of the present invention, the host 1004 is anetwork-enabled device and is configured to communicate with at leastone remote device (e.g., 1006, 1008, 1010) via a wide area network (WAN)1000. For example, the host 1004 may be configured to store/retrieveindividual state profiles (e.g., PAIID) on/from a remote database (e.g.,a “cloud”) 1010, and/or share individual state profiles (e.g., PAIID)with other network-enabled devices (e.g., 1006, 1008). The profilescould be stored for future retrieval, or shared in order to allow otherdevices to determine an individual's current state. As discussed above,the host 1004 may gather self-reporting data that links characteristicsof the individual to particular states. By sharing this data with otherdevices, those devices can more readily determine the individual'scurrent state without having to gather (from the individual)self-reporting (or calibration) data. The database 1010 could also beused to store historical states, or states of the individual over aperiod of time (e.g., a historical log of the individual's priorstates). The log could then be used, either alone or in conjunction withother data, to determine an individual's state during a relevant time ortime period (e.g., when the individual was gaining weight, at the timeof an accident, when performing a discrete or specific action, etc.), orto determine indications as to psychological aptitude or fitness toperform certain functions where, for example an individual's state is ofcritical importance, such as, but not limited to piloting a plane,driving a heavy goods' vehicle, or trading instructions on financial orcommodities exchanges.

The state log could be further utilized to generate a state “bot” whichis an agent of the individual capable of being distributed over anetwork to look for information on behalf of the individual which islinked to a particular thing the individual has an “interest” in, orwishes to be informed of, either positive or negative, conditional ontheir being in that particular state.

In an alternate embodiment, information, such as historical logs orindividual state profiles (e.g., PAIID) are also, or alternatively,stored on a memory device 1024 on the host 1004 (see FIG. 11). In thisembodiment, the host 1004 may include a transceiver 1032, a processor1022, a display 1026, and at least one application 1028 (see FIG. 11),all of which function the same as similar components depicted in FIG. 2.The host 1004 may also include at least one microphone and/or at leastone camera 1030 configured to acquire audio/video from/of the individual(e.g., a driver of a vehicle). As previously discussed, the audio/videocan be used to determine at least one state of the individual. Forexample, the individual's speech and/or facial features, either alone orin combination with other data (e.g., heart rate data acquired fromsensors on the steering wheel, etc.), could be analyzed to determine atleast one state of the individual. The state can then be used to performat least one action. In one embodiment of the present invention, thestate is used to determine whether a request (e.g., command, etc.) fromthe individual should be carried out, and if so, whether other actionsshould also be performed (e.g., limiting speed, providing a warning,etc.). For example, if a driver of a vehicle instructs the vehicle tostart, the vehicle (or host operating therein) could provide the driverwith a warning if it is determined that the driver is tired, or couldinitiate auto-pilot mode if it is determined that the driver is impaired(e.g., under the influence). In another example, an airline pilot couldbe asked to provide a response as to how they're feeling, and dependenton how the pilot responds, both by nature of the content of their replyand its analyzed state, air traffic control can seek to take theappropriate action to seek to ensure the safety of the plane. In thiscase, and cases of a similar nature or context failure to provide anykind of response would provide an alert which might indicate either thatthe pilot didn't wish to respond (which is information in itself) or wasnot in a situation to respond.

It should be appreciated that in embodiments where the individual isresponding to a thing, the thing could be anything in close proximity tothe individual, including a person (or a person's device (e.g.,smartphone, etc.)), a place (e.g., based on GPS coordinates, etc.), orcontent shown to the user (e.g., subject matter of textual data like anemail, chat message, text message, or web page, words included intextual data like an email, chat message, text message, or web page,subject matter of video data, subject matter of audio data, etc.). The“thing” or data related thereto can either be provided by the networkdevice to the Web host, or may already be known to the Web host (e.g.,when the individual is responding to web-based content provided by theWeb host, the emotional response thereto could trigger additional data,such as an advertisement).

A method of carrying out the present invention, in accordance with oneembodiment of the present invention, is shown in FIG. 5. Starting atstep 500, biometric data is received at step 502. As discussed above,the biometric data can be at least one physical and/or biologicalcharacteristics of an individual, including, but not limited to, heartrate, blood pressure, temperature, breathing rate, facial features,changes in speech, changes in eye movement and/or dilation, and chemicalcompositions (in blood, sweat, saliva, urine or breath). The biometricdata is then used to determine a corresponding emotion at step 504, suchas happiness, anger, surprise, sadness, disgust, or fear. At step 506 adetermination is made as to whether the emotion is the individual'scurrent state, or whether it is based on the individual's response to athing (e.g., a person, place, information displayed to the individual,etc.). If the emotion is the individual's current state (step 508), thenweb-based data is selected based on the individual's current emotionalstate at step 512. If, however, the emotion is the individual's responseto a thing (step 510), then web-based data is selected based on theindividual's emotional response to the thing at step 510. The selectedweb-based data is then provided to the individual at step 514, stoppingthe process at step 516.

It should be appreciated that the present invention is not limited tothe method shown in FIG. 5, and methods that includes additional, fewer,or different steps is within the spirit and scope of the presentinvention. For example, at step 512, the web-based data may be selectedusing emotion data (or emotion-thing data) and interest data. By way ofanother example, in step 514, the selected content (e.g., web-baseddata, text message, email, etc.) may also (or alternatively) be providedto a third person, such as a legal guardian of the individual, a familymember of the individual, a medical staff member (if the individual isin the hospital), emergency response (if the individual is not in thehospital), etc. The present invention is also not limited to the stepsrecited in FIG. 5 being performed in any particular order. For example,determining whether the emotion is the individual's current state or theindividual's response to a thing may be performed before the receptionof biometric data.

While biometric data, and the like, can be very simple in nature (e.g.,identifying the characteristic being measured, such as blood pressure,and the measured value, such as 120/80), it can also be quite complex,allowing for data to be stored for subsequent use (e.g., creatingprofiles, charts, etc.). For example, in one embodiment of the presentinvention, as shown in FIG. 6, biometric-sensor data may includedetailed data, such as reference-id (technical unique-identify of thisdatum), entity-id (a user, team, place word or number, device-id),sensor-label (a string describing what is being measured), numeric-value(integer or float), and/or time (e.g., GMT UNIX of when the measurementwas taken). As shown in FIG. 7, emotional-response data may includereference-id (technical unique-identifier of this datum), entity-id (auser, team, place word or number, device-id), emotion-label (a stringthat recognizes this as an emotion), time (e.g., GMT UNIX timestamp whenthis record was created), emotional-intensity (numeric-value), and/ordatum-creation data (a technical reference to what system created thisdatum and/or which data was used to create this datum). As shown in FIG.8, emotion-thing data may include reference-id (technicalunique-identifier of this datum), entity-id (a user, team, place word ornumber, device-id), emotion-reference (a reference to a specific emotiondocumented elsewhere), thing-reference (a reference to a specific thingdocumented elsewhere), time (e.g., GMT UNIX timestamp when this recordwas created), correlation-factor (numeric-value representing a scale ofcorrelation, such as a percent), emotional-intensity (numeric-value),and/or datum-creation data (a technical reference to both what systemcreated this datum and/or which data was used to create this datum). Asshown in FIG. 9, thing data may include reference-id (technicalunique-identifier of this datum), entity-id (a user, team, place word ornumber, device-id), thing-reference (a reference to specific “thing”documented elsewhere), time (e.g., GMT UNIX timestamp when this recordswas created), correlation-factor (numeric-value representing a scale ofcorrelation, such as a percent), and/or datum-creation data (a technicalreference to both what system created this datum and/or which data wasused to create this datum). It should be appreciated that the presentinvention is not limited to the data strings shown in FIGS. 6-9, andother methods of communicating said data is within the spirit and scopeof the present invention.

A method of carrying out the present invention, in accordance withanother embodiment of the present invention, is shown in FIG. 12.Starting at step 1200, a request is received from a user at step 1202.As discussed above, the request may include a question asked by the user(dictating a response) or a command provided by the user (dictating theperformance of an action). The request (or other biometric data) is thenanalyzed to determine the user's current state at step 1204, such as acorresponding emotional state, mood, physical state, and/or mentalstate. At step 1206, the user's current state is used to determinationwhether a particular action should be performed. For example, if theuser's state is normal, then the requested action (e.g., the actionrequested at step 1202) is performed at step 1210, ending the method atstep 1220. If the user's state is abnormal, but not alarming (e.g.,angry), then a warning may be provided at step 1212. If the user's stateis abnormal and alarming (e.g., intoxicated), then a different action(e.g., an action that is different from the one requested at step 1202)may be performed at step 1208. If a warning is provided at step 1212, ora different action is performed at step 1208, then a determination ismade at steps 1220 and 1214, respectively, as to whether the requestedaction (e.g., the action requested at step 1202) should be performed. Ifthe answer is YES, then the requested action is performed at step 1210,ending the method at step 1220. If the answer is NO, then no furtheraction is taken, ending the method at step 1220.

It should be appreciated that the present invention is not limited tothe method shown in FIG. 12, and methods that includes additional,fewer, or different steps is within the spirit and scope of the presentinvention. The present invention is also not limited to the stepsrecited in FIG. 12 being performed in any particular order.

Having thus described several embodiments of a system and method forusing biometric data to determine at least one state, and using the sameto perform a particular result (i.e., personalization preferenceoptimization), it should be apparent to those skilled in the art thatcertain advantages of the system and method have been achieved. Itshould also be appreciated that various modifications, adaptations, andalternative embodiments thereof may be made within the scope and spiritof the present invention. The invention is solely defined by thefollowing claims.

What is claimed is:
 1. A method for using at least self-reported and biometric data to determine at least one state of a user and to perform at least one action in response to said determined state, said determined state being one of an emotional state and a physical state of said user, comprising the steps of: receiving by a processor first biometric data of said user, said first biometric data being received from at least one sensor; using at least said first biometric data to infer a first state of said user at a time that said first biometric data is received; receiving by said processor self-reporting data from said user, said self-reporting data being received immediately after said first biometric data is received and allowing said user to explicitly identify their current state, thereby indicating whether said inferred state is correct; storing said first biometric data and said self-reporting data in a memory to associate said first biometric with said self-reporting data; receiving by said processor a request from said user to perform an action, said request being received after said biometric data; using at least said first biometric data, said request, and said self-reporting data to determine a second state of said user; and performing said at least one action based at least on said second state, wherein said at least one action is said requested action if said second state is normal and therefore consistent with said performance of said requested action and said at least one action is a second action if said second state is abnormal and therefore inconsistent with said performance of said requested action, said second action including notifying said user of (i) said second state and (ii) an alternate action that could be performed instead of said requested action, wherein said alternate action is an unrequested action and different from said requested action, said alternate action being performed if said user indicates in the affirmative that said alternate action should be performed, otherwise, in the absence of said affirmative indication, performing said requested action.
 2. The method of claim 1, wherein said first biometric data includes at least one change in facial muscle expressions, breathing rates, speech pitch, auditory changes, and body movement.
 3. The method of claim 1, wherein said first state comprises one of happiness, sadness, surprise, anger, disgust, fear, euphoria, attraction, love, calmness, amusement, excitement, tiredness, hunger, thirst, well-being, sick, failure, triumph, interest, enthusiasm, animation, reinvigoration, and satisfaction.
 4. The method of claim 1, wherein said step of receiving said self-reporting data from said user comprising receiving at least one of audible data from said user via a microphone and tactile data from said user via an input device.
 5. The method of claim 1, further comprising the step of receiving by said processor ambient data comprising at least one of location of said user, temperature at said location, altitude at said location, and time of day, wherein said processor further uses said ambient data to determine said second state of said user at said time that said second biometric data is received.
 6. The method of claim 1, wherein said step of using said first biometric data to infer said first state further comprises comparing said first biometric data to known correlations between different biometric data and different states to infer said first state.
 7. The method of claim 1, wherein said second state is further determined from other biometric data and other self-reporting data, where said individual ones of said other self-reporting data are received immediately after individual ones of said other biometric data.
 8. The method of claim 1, further comprising a step of performing both said requested action and said alternate action.
 9. The method of claim 1, further comprising the step of prompting said user to explicitly identify their current state.
 10. The method of claim 9, wherein said step of prompting said user to explicitly identify their current state further includes providing said user with said first inferred state.
 11. The method of claim 1, wherein said step of using at least said first biometric data and said self-reporting data to determine said second state comprises checking whether said explicitly identified state matches said first inferred state.
 12. The method of claim 1, wherein second biometric data is extracted from said request and used to along with said first biometric data and said self-reporting data to determine said second state.
 13. The method of claim 1, further comprising the step of providing said first state to said user prior to receiving said self-reporting data.
 14. The method of claim 13, wherein said self-reporting data comprises at least one of confirming, denying, and correcting said first state.
 15. The method of claim 1, further comprising the step of determining an intensity level for said second state, said intensity level being a numerical value corresponding to a strength of said second state, said action being further based on at least said intensity level.
 16. A system for determining at least one state of a user, said at least one state being one of an emotional state and a physical state of said user, comprising: at least one processor; and at least one memory device in communication with said processor and for storing machine readable instructions, said machine readable instructions being adapted to perform the step of: receiving first biometric data of said user, said first biometric data being received from at least one sensor; using at least said first biometric data to infer a first state of said user at a time that said first biometric data is received; receiving self-reporting data from said user, said self-reporting data being received immediately after said first biometric data is received and allowing said user to explicitly identify their current state, said state being said first state or a third state; storing said first biometric data and said self-reporting data in said at least one memory device so that said first biometric data is linked to said self-reporting data; receiving a request from said user to perform an action, said request being received after said self-reporting data has been received; using at least said first biometric data, said request, and said self-reporting data to determine a second state of said user; and performing at least one action, said action being based on at least said second state, wherein said action is said requested action if said second state is normal and therefore consistent with said performance of said requested action and said at least one action is a second action if said second state is abnormal and therefore inconsistent with said performance of said requested action, said second action including then notifying said user of (i) said second state and (ii) an alternate action that could be performed instead of said requested action, wherein said alternate action is an unrequested action and different from said requested action, said alternate action being performed if said user indicates in the affirmative that said alternate action should be performed, otherwise, in the absence of said affirmative indication, performing said requested action.
 17. The system of claim 16, wherein said first biometric data includes at least one change in facial muscle expressions, breathing rates, speech pitch, auditory changes, and body movement.
 18. The system of claim 16, further comprises at least one of a microphone and a keyboard for receiving said self-reporting data from said user.
 19. The system of claim 16, wherein said second state is further determined from other biometric data and other self-reporting data from at least one other user.
 20. The system of claim 16, wherein said sensor is a microphone.
 21. The system of claim 16, further comprising an Internet of Things (IOT), said IOT comprising said at least one processor and said at least one memory device.
 22. The system of claim 16, further comprising prompting said user to identify their current state.
 23. The system of claim 16, wherein said machine readable instructions are further adapted to perform said requested action in addition to said alternate action.
 24. The system of claim 23, wherein said digital data is further selected from said plurality of digital data based on at least one thing in a proximity of said user at said time that said request is received said requested action is performed before said alternate action.
 25. The method of claim 16, wherein said machine readable instructions are further adapted to provide said first state to said user, said self-reporting data comprising at least one of confirming, denying, and correcting said first state.
 26. The method of claim 16, wherein said machine readable instructions are further adapted to determine a strength of said second state, said action being further based on at least said strength of said second state.
 27. A system that uses artificial intelligence (AI) to determine a state and to use said state to perform at least one action, comprising: at least one server in communication with a wide area network (WAN); a mobile device in communication with said at least one server via said WAN, said mobile device comprising: at least one processor for downloading machine readable instructions from said at least one server; and at least one memory device for storing said machine readable instructions that are adapted to perform the step of: receiving first biometric data from said user, said first biometric data being acquire using at least one sensor and analyzed to infer a first state at a time that said first biometric data is received; receiving self-reporting data from said user, said first self-reporting data being received immediately after said first biometric data is received and allowing said user to provide the current state, thereby indicating whether said inferred state is correct; receiving a request from said user to perform an action, said request being received after said self-reporting data has been received; and wherein at least said first biometric data, said request, and said self-reporting data are used to determine a second state at a time that said request is received; wherein at least one action is performed after said second state is determined and in response to said request, said at least one action (i) being based on said second state, (ii) being said requested action if said second state is normal and therefore consistent with said performance of said requested action, and (iii) being a second action if said second state is abnormal and therefore inconsistent with said performance of said requested action, said second action including then notifying said user of (i) said second state and (ii) an alternate action that could be performed instead of said requested action, wherein said alternate action is an unrequested action and different from said requested action, said alternate action being performed if said user indicates in the affirmative that said alternate action should be performed, otherwise, in the absence of said affirmative indication, performing said requested action.
 28. The system of claim 27, wherein using at least said first biometric data, said result, and said self-reporting data to determine said second state further comprises (i) using said self-reporting data to check whether said first inferred state should be adjusted, and (ii) using at least the same and said result to determine said second state. 